Entropy Lower Bounds Related to a Problem of Universal Coding and Prediction
نویسندگان
چکیده
Second order lower bounds for the entropy function expressed in terms of the index of coincidence are derived. Equivalently, these bounds involve entropy and Rényi entropy of order 2. The constants found either explicitly or implicitly are best possible in a natural sense. The inequalities developed originated with certain problems in universal prediction and coding which are briefly discussed.
منابع مشابه
Redundancy-Related Bounds on Generalized Huffman Codes
This paper presents new lower and upper bounds for the compression rate of optimal binary prefix codes on memoryless sources according to various nonlinear codeword length objectives. Like the most well-known redundancy bounds for minimum (arithmetic) average redundancy coding — Huffman coding — these are in terms of a form of entropy and/or the probability of the most probable input symbol. Th...
متن کاملUniversal Finite Memory Coding of Binary Sequences
This work considers the problem of universal coding of binary sequences, where the universal encoder has limited memory. Universal coding refers to a situation where a single, universal, encoder can achieve the optimal performance for a large class of models or data sequences, without knowing the model in advance, and without tuning the encoder to the data. In the previous work on universal cod...
متن کاملA Lower-Bound for the Maximin Redundancy in Pattern Coding
We show that the maximin average redundancy in pattern coding is eventually larger than 1.84 ( n logn )1/3 for messages of length n. This improves recent results on pattern redundancy, although it does not fill the gap between known lowerand upper-bounds. The pattern of a string is obtained by replacing each symbol by the index of its first occurrence. The problem of pattern coding is of intere...
متن کاملUniversal Quantization for Separate Encodings and Joint Decoding of Correlated Sources
We consider the multi-user lossy source-coding problem for continuous alphabet sources. In a previous work, Ziv proposed a single-user universal coding scheme which uses uniform quantization with dither, followed by a lossless source encoder (entropy coder). In this paper, we generalize Ziv’s scheme to the multi-user setting. For this generalized universal scheme, upper bounds are derived on th...
متن کاملIRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Universal Quantization for Separate Encodings and Joint Decoding of Correlated Sources
We consider the multi-user lossy source-coding problem for continuous alphabet sources. In a previous work, Ziv proposed a single-user universal coding scheme which uses uniform quantization with dither, followed by a lossless source encoder (entropy coder). In this paper, we generalize Ziv’s scheme to the multi-user setting. For this generalized universal scheme, upper bounds are derived on th...
متن کامل